Balancing Time-to-Market and Quality in Embedded Systems

Finding a balance between the time-to-market and quality of a delivered product is a daunting task. The optimal release moment is not easily found. We propose to use historical project data to monitor the progress of running projects. From the data we inferred a formula providing a rough indication of the number of defects given the effort spent thus far (effort-to-defect formula). Furthermore we provide a worst case bound to the allowed number of residual defects at the end of a project in order to achieve the required level of quality. For this purpose we slightly modified a well-known reliability growth model by Bishop and Bloomfield. It turned out that the software in Philips’ MRI scanners has a defect rate of 1 per 1,175 device-years of observation. This coincides with the second highest safety integrity level (SIL3) as defined in the IEC 61508 standard. The highest level (SIL4) is only attainable by applying redundancy. Finally, we combine the effort-to-defect formula with the reliability growth model to monitor the progress of a project and to determine when the required level of quality will be 1 This work has been carried out as a part of the DARWIN project at Philips Healthcare under the responsibilities of the Embedded Systems Institute. This project is partially supported by the Dutch Ministry of Economic Affairs under the BSIK program. Furthermore, this research received partial support by the Dutch Joint Academic and Commercial Quality Research & Development (Jacquard) program on Software Engineering Research via contract 638.004.405 EQUITY: Exploring Quantifiable Information Technology Yields and via contract 638.003.611 Symbiosis: Synergy of managing business-IT-alignment, IT-sourcing and offshoring success in society. reached. We show that a common fault distribution, the Rayleigh model, is not necessarily the best model for predicting the number of residual defects in the system. Using a well-known data analysis approach called Exploratory Data Analysis (EDA) we obtained an alternative model based on the Normal curve. We have evaluated the Rayleigh model and our model based on the Normal curve at Philips Healthcare MRI. The Normal curve predicts defects over time better compared to the Rayleigh model in the case of Philips Healthcare MRI. Furthermore, time series models (ARIMA) are also useful for accurately describing the defect trend, but are not suitable for long-term predictions. Finally, cost estimation models (COCOMO) lack the predictive capabilities of models fitted on the data using EDA. Their capabilities are limited compared to a model derived from data which reflects the constitutional knowledge of the actual realization of systems within a specific company. However, they can still be used to advantage when there is limited or no data available to use as a basis for ifitng from gutfeel to order of magnitude.